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Qj ' Abstract 

1"^ . A sender wishes to broadcast a message of length n over an alphabet to r users, where 

^t| I each user i, 1 < i < r should be able to receive one of rrii possible messages. The broad- 

cast channel has noise for each of the users (possibly different noise for different users) , who 
cannot distinguish between some pairs of letters. The vector (7711,7712, . . . ,7nr)(„) is said to 
be feasible if length n encoding and decoding schemes exist enabling every user to decode his 
tyj ' message. A rate vector {Ri, R2, . . . , Rr) is feasible if there exists a sequence of feasible vectors 

^, ' {mi, 1712, ■ ■ ■ , 'rnr)(n) such that Ri = lim„H_>oo °^^™' , for all i. 

We determine the feasible rate vectors for several different scenarios and investigate some 
of their properties. An interesting case discussed is when one user can only distinguish between 

Q>^ ' all the letters in a subset of the alphabet. Tight restrictions on the feasible rate vectors for 

>0 ■ some specific noise types for the other users are provided. The simplest non-trivial cases of two 

users and alphabet of size three are fully characterized. To this end a more general previously 

I _,^ ^ known result, to which we sketch an alternative proof, is used. 

f*"^ ' This problem generalizes the study of the Shannon capacity of a graph, by considering 

^^ . more than a single user. 
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1 Introduction 

A sender has to transmit messages to r-users, where the user number i should be able to receive 
any one of nii messages. To this end, the sender broadcasts a message of length n over an alphabet 
T, of size k. Each user i has a confusion graph Gi on the set of letters of S, where two letters 
a,b G T, are connected if and only if user i cannot distinguish between a and b. The sender and 
users can agree on a (deterministic) coding scheme. For each possible values Oj of the messages, 
^ ^ cLi < rrii, the scheme should enable the sender to transmit a string of length n over S so 
that each user i will be able to recover Oj. The vector of a scheme for length n messages is 
(m-i, 7772, . . . , 777,.). The rate vector of a sequence of schemes is the limit 
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assuming the limit exists for this sequence. Our objective is to study which vectors and which 
rate vectors are feasible for a given set of confusion graphs Gj. This seems to be difficult even for 
relatively small cases, and reveals some interesting phenomena. Note that this problem generalizes 
the problem of computing the Shannon capacity of a graph which was first considered by Shannon 
in [10] , In the case of a single user (i.e. a single confusion graph G), the maximum feasible rate 
is precisely log2 c{G) where c{G) denotes the Shannon capacity of G. 

Investigating the feasible rate vectors for a given set of confusion graphs raises another inter- 
esting question. What is the maximum capacity of the channel for all users together? The total 
capacity can be measured as the sum of rates for each user, which we refer to as the total rate. 
This sum encapsulates the usability of the channel. 

The individual rates we consider in this paper are sometimes referred to as private rates. Sim- 
ilarly, there is an analogue question about the common rate, determining how much information 
could we use if we require all users to receive the same message. This question is outside the scope 
of our work, but for completeness we refer the reader to [5j for more information and results. 

1.1 Initial Observations 

The relation between the described problem and the Shannon capacity leads to the following upper 
bound on the users' rates and hence for the total rate as well. 

Proposition 1. Given r users whose confusion graphs are Gi,G2, . . . ,Gr, a feasible rate vector 
(i?i, i?2, • • • , Rr) must satisfy Ri < log2 c(Gj) for every 1 < i < r, hence Yll=i Ri — Yll=i lo§2 c(Gj). 

Although in practice we have several users, their total rate cannot exceed the possible rate 
of a single user who shares all their information. Given a set of confusion graphs Gi, . . . , G^, let 
G = nl^iGi be the graph over the same alphabet S, where o, 6 E S are connected in G if and 
only if they are connected in Gj for every i. The confusion graph G represents the information all 
users have together and therefore can bound their total rate as follows. 

Proposition 2. Given r users whose confusion graphs are Gi, G2, . . . , Gr, any feasible rate vector 
(i?i, i?2, ... ,Rr) must satisfy Yl'i=i Ri < log2 c{<^'i=iGi)- 

An important simple property of the feasible rate vectors is convexity, which is often referred 
to as time sharing. Informally, the messages we broadcast can be shared between two or more 
broadcasting schemes, where each part corresponds to a different scheme. This property can be 
stated formally as follows. 

Proposition 3. Let Gi, G2, . . . , Gr be the confusion graphs for r users. Given two feasible rate 
vectors R,R and a £ [0, 1], the rate vector aR + (1 — a)R is also feasible. 

Proof. Since both R and R are feasible rate vectors, each has some corresponding encoding 
scheme. Our new encoding scheme would be to use the encoding scheme corresponding to R in 



the first an coordinates and the one corresponding to R in the remaining (1 — a)n coordinates. 
The resulting rate vector is precisely aR + (1 — a)R , as required. D 

Corollary 4. Let Gi,G2, ■ ■ ■ ,Gr be the confusion graphs for r users. Given xi,X2, ■ ■ ■ ,Xr S [0, 1] 
so that Yll=i ^i ~ -*-' ^^^ '''^^^ vector {xi ■ log2 c(Gi), . . . , x^ • log2 c{Gj.)) is feasible. 

Proof. For every 1 < i < r, the rate vector consisting of rate log2 c(Gj) for the i'th user and zero 
rate for all other users is trivially feasible. The result thus follows by Proposition [3l D 

1.2 Previous Results 

The problem of simultaneous communication in noisy channels was previously studied in the 
theory of broadcast channels (see [3] and its references). For some scenarios, such as the one we 
will describe shortly, a full characterization of all feasible rate vectors was found (see [8] and [9| ) . 
This scenario is described here fully as it is used in some of our proofs and as we also provide a 
sketch for an alternative proof for it. 

Let Sfc = {cTi, (T2, . . . , CT/c} be an alphabet of size k and let Gi,G2, ■ ■ ■ ,Gr be the confusion 
graphs for the r users, where each confusion graph is a disjoint union of cliques. Given user i, 
one can view his noise as receiving yi = fi{x) whenever x is transmitted, where fi{x) : S^ i— )■ 
{1, 2, . . . ,ii} is the index of the clique which x belongs to and ii is the number of cliques in the 
i'th user's confusion graph. Note that we consider isolated vertices as cliques of size one and hence 
fi is well defined up to the order of the cliques. 

Definition 1. Given a probability distribution p overT,k and a subset of the users I Q {1, 2, . . . ,r}, 
let H^p^{{Yi}i^i) be the binary entropy of the random variables {l^}jg/ where Yi = fi{X) and X 
is the random variable distributed over T,k according to p. 

For each subset of the users / C {1, 2, . . . , r}, the alphabet S^ can be partitioned into s < k 
disjoint parts {Ai, . . . , Ag) according to what these users receive ({/i(x)}jg/). These users cannot 
distinguish between different letters from the same part Aj, so their joint information when sending 
a letter X from S^ according to the probability distribution p can be computed as 

%)({^.}.e/) = - E P^[^ ^ ^^l • log2Pr[X G A,] . 

l<j<s 

Therefore in a sense that will be made precise later, when using only messages in which the letters 
are distributed according to some distribution p, we expect no subset / of users to have total rate 
which exceeds ff(p)({li}je/). 

The following theorem, for which we sketch an alternative proof, provides the full characteri- 
zation of all feasible rate vectors. 

Theorem 5 ([9]). Let Gi,G2, ■ ■ ■ ,Gr be the confusion graphs for r users over the alphabet S^ = 
{o"i, o"2, . . . , CTk}, where each confusion graph is a disjoint union of cliques. Using the notations and 



definitions above, a rate vector {Ri, R2, ■ ■ ■ , Rr) is feasible if and only if there exists a probability 
distribution p = (pi, . . . ,pk) over T,k so that for every subset / C {1, 2, . . . , r} of the users, 

Y,R^<H^p)i{Yi}^eI)■ 

An interesting special case of the above theorem is the symmetric dense scenario of r = /c 
users, where the confusion graph Gi of user i is a chque over S^ — {cjj}. In other words, user i 
only distinguishes the letter ctj from all other letters. When decoding, the relevant information 
for user i is only the locations of at in the transmitted message. 

Corollary 6. For every fixed k > 3, (— ^— , • • • , °^^ ' ) is a feasible rate vector over the alphabet 
Sfe = {o"i,o"2) • • • jCTfc}, when each confusion graph Gi is a clique on S^ — {ci}. 

Corollary [H] indicates the possible gain of encoding schemes for several users simultaneously. 
The total rate here is log2 k, whereas using convexity with encoding schemes for single users cannot 
exceed a total rate of 1 (as this is the maximum rate for each single user; the Shannon capacity 
c(Gj) is precisely 2 for each 1 < i < k). However, in some cases there is no such gain. Several 
examples are discussed in what follows. 

1.3 Our Results 

For simplicity we omit all floor and ceiling signs whenever these are not crucial. 

Let Sfc = {(Ti, (72, . . . , o-fc} be an alphabet of size k and let S^^ = {ai, . . . , ad} denote the set of 
the first d letters of S^, where 2 < d < k. Consider the case where user 1 has a confusion graph 

Gi = (Sfc, {a6 I a G Sfe A 6 G Sfc - Sd}), 

meaning the complete graph over S^ minus a clique over S^. The Shannon capacity of such graphs 
is easily shown to be c(Gi) = d, hence the maximum rate of user 1 is at most log2 d. The following 
results indicate, that for two different confusion graphs of user 2, nothing can be gained beyond 
convexity of single user encoding schemes. We need the following definition. 

Definition 2. A rate vector is optimal if no user can increase his rate while the other user 
maintains the same rate. 

Note that the total rate of such optimal rate vectors does not necessarily reach the maximum 
total rate possible. 

Theorem 7. In the scenario described above for 2 < d < k, when user 2 has the empty confusion 
graph G2 = (Sfci O), the rate vectors (a log2 d,{l — a) log2 k) for a G [0, 1] are optimal. 

Theorem 8. In the scenario described above for 2 < d < ^ii, when user 2 has the complement 
confusion graph, that is 

G2 = Gi = (Sfe, {ab \a,b £ ^d}), 

the rate vectors (a log2 d,{l — a) log2(A; — d + 1)) for a G [0, 1] are optimal. 



Finally, we provide a full characterization of all feasible rate vectors for all scenarios containing 
two users and alphabet of size three (Propositions [T5 t 1161 and I17p . 

1.4 Organization 

The rest of the paper is organized as follows. In Section[2]we present a sketch of an alternative proof 
to the characterization of the feasible rate vectors for the first scenario, where each confusion graph 
is a union of disjoint cliques, (Theorem [5]) , and demonstrate how combining encoding schemes for 
many users can sometimes outperform convexity (Corollary [6]). Section [3] deals with the second 
family of confusion graphs described above in which convexity yields the optimal rate vectors 
(Theorems [7] and [8|) . Combining these results, one can characterize all feasible rate vectors for all 
scenarios involving two users and alphabet of size three. In Section S] we elaborate in more details 
on this analysis. The final Section [5] contains some concluding remarks and open problems. 

2 Unions of disjoint cliques - outperforming convexity 

We consider the case where the confusion graph of each user i is a disjoint union of cliques. This 
case is of special interest as a full description of all feasible rate vectors was known and it is 
deterministic in the sense that given the transmitted letter, we can transform it deterministically 
to the different symbols that each user receives. Moreover, choosing specific confusion graphs, 
it demonstrates how the maximum possible total rate can be achieved by combining schemes for 
many users (and only this way), even when the confusion graphs are nearly complete. 

2.1 An alternative proof of Theorem [5] (sketch) 

Let Gi,G2, ■ ■ ■ ,Gr be a set of confusion graphs for r users over the alphabet S^, where each Gj 
is a disjoint union of cliques. Given a subset of the users I C [r] (where [r] = {1,2,..., r}), the 
following definition and Lemma connects between the possible number of messages to these users 
and their binary entropy, when restricted to a specific distribution of the messages 

Definition 3. Given a probability distribution p over S^ and a subset of the users I CI [r], 
let Nrp\(n; {Yi}i^j) denote the number of possible (different) messages for these users under the 
restriction that each message is originated in a length n message over S^ in which ai appears piU 
times. 

Lemma 9. 

-^ < iV(p)(n;{y,},e,) < 2^(p)(i^>}>^^)". 

The proof of Lemma [9] is a simple consequence of Stirling's formula, which is left to the reader. 

Upper bound. Given a scheme of a fixed length n which realizes {mi,m2, ■ ■ ■ ,mr) messages for 
the r users, one can divide it into families according to the number of appearances of the letters 



CTj in each message. As there are only k letters and all the messages are of length n, there are 
at most n different families. Given the probability p = {pi,P2, ■ ■ ■ ,Pk) corresponding to the 
largest family, Lemma |9] therefore indicates that 

Hm^ < iV(^)(n;{y,},e/) < 2^(.)«^^i-^)^ 
Recall that Ri = lim„i_>.oo log2 rrii/n and hence 

Yr. = hm y'^!^^= lin, l^gilki!^ 

■'- — ' ni-4-oo ■'- — ' n ni-^oo n 

^ ,. H(^p){{Yi}i^i)n + {k-l)log2n 
< lim 

ni->(X) n 

= H(^p){{Yi}i(zi) 

as required. D 

Remark: Formally, the popular probability distribution p = {pi,p2, ■ ■ ■ ^Pr) depends on n but 
one can take a subsequence for which it converges to a single vector p, justifying the computation 
above. A similar argument can be used in the following lower bound proof, justifying it for any 
probability distribution, even one containing irrational probabilities. 

Remark: Similar arguments are sometimes referred to as type counting and could be found, for 
example, in the book by Csiszar and Korner [5]. They also provide a proof for a claim similar to 
Lemma [H 

Lower bound. Let p = {pi, . . . ,pk) be a probability distribution over E^ and fix {Ri,R2, . . . , Rr) so 
that for any subset of the users / C [r], J2iei ^* — -^(p)({^*}«e/)- Given some large n, we set the 
number of messages rui for each user i to be rrii = -j^ (which clearly satisfies lim„i_j.oo °^^™' = 
lim„H_>oo '" n °^^" = Ri)- By Lemma O for every subset of the users / C [r], 

\\mi = li^^FTf = „(fe+2)|/| 
iei iei 

— 11^+'^ ~ n^ 

Our encoding scheme will use only messages in which the letters of S^ are distributed according 
to p. For each user i there are Ni„\{n; {Yi}) different messages that he can identify. We randomly 
divide them into mj families J^j,i, . . . ,J^i,mi, where each family represents a different message for 
user i. When the message x G S^ is transmitted, user i receives y^ = fi{x) = fi{xi) ■ ■ ■ fi{xn) and 
decodes the message j, the single j G [mj\ for which y^ G Fi^j. 

In order to complete the proof we show the described encoding scheme is valid for some 
selection of families -T^ij- Such a scheme is valid if for every set of messages {ji G [w-jjliew there 
exists a message x so that for every user i G [r] , y^ G J-", j- . . 



Given fixed messages ji,J2, ■ ■ ■ ,jr for the r users, using the extended Janson inequahty (c.f., 
e.g., [2], Chapter 8) and ([T]) one can show that the probabihty that there exists no valid message 
as required is less than ■^. As there are 111=1 "^j ^ ^" distinct choices for messages ji, . . . , jV, the 
assertion of Theorem [5] follows by the union bound. D 

Remark: A full citation of the extended Janson inequality can be found in Appendix[Al together 
with a more detailed description of how it is used here. 

2.2 Proof of Corollary M 

This corollary of Theorem [5] is for the symmetric dense case where there are r = k users, and 
each user i distinguishes a single letter from all other letters. This simple case demonstrates how 
the maximum rate can be achieved only by mixing the messages for all the users. Since each 
confusion graph Gi is a clique on S^ — {o",}, the Shannon capacity of this graph is 2 and therefore 
the maximal rate for any single user is 1. However, the theorem shows that indeed a total rate of 
log2 k can be achieved, and obviously this is best possible. 

Let A; > 3 be fixed. In order to prove the theorem, a probability distribution p is required so 
that for every subset of the users / ^ [r] = [A;] , 

^i?, = |/|i^<i7(,)({y.W). (2) 

i&I 

Let us consider the uniform probability distribution p, pi = 1/k for every letter Cj. When con- 
sidering all users together, the random variables {^jiew indicate the exact letter x that was 
transmitted. Therefore, since the entropy is exactly log2 /c, -ff(p)({^}ie[r]) = log2 ^ which indeed 
satisfies ([2]) as X^jgu.i Ri = r °^^ = log2 k. 

Given a subset of the users / C [r], the random variables {l^}jg/ indicate which letter x was 
transmitted if x = Uj for i £ I or alternatively, that some other letter was transmitted. Hence its 
binary entropy satisfies 



i^(p)({^.W) = |/|^ + ^^log2 

k 



logo k k — \I\ , k 

— \ — logo r^ 

k k ^^k-\I\ 

log2/c 



> \i\-^=yR 

h ^ — ^ 



thus ([2]) holds for every subset of the users I C [r], as required. D 

3 A clique minus a clique - convexity is everything 

We consider the case where the confusion graph Gi of the first user is the complete graph on 
k vertices minus a clique on d vertices. We give an upper bound on the rate of the other user 
for both the empty confusion graph and for Gi. In both cases, the results imply that optimal 



encoding can be achieved by convexity, that is nothing can be gained by encoding the messages 
together. In order to prove Theorems \7\ and [HI we need the fohowing lemmas whose proofs are 
provided in Appendix [Bl 

Lemma 10. Given a,b gN s.t. 2 <b < a and xi > X2 > • • • > a;;, > 0, 

{a-b+ l)a^°^^ ""+ Yl "'°*^' ""^ - "'°^' ^'^'" ""' ■ 

iG[fe-l] 

Remark: Define here a}°^'>0 — q hence we allow x;, to be 0. 

Lemma 11. Given 2 < d < fc G N and a set G QTi"^, we define Q' = Q r\Y7^. If Q is closed under 
replacing each CTi with Gj for any i > d and j G [A;], then either \G\ = \G'\ = or 

logfciai <loga\G'\. 

Lemma 12. Given 2 < d < k £ N s.t. d < ^ and a set G (^ E^, define G' = GnT.]^ and 
G" = {f {91)1(92) • • • fi9n) \ 9 ^ G} where f{ai) = crmax{j,4- If G is closed under replacing each 
ai with Gj for any i > d and j G [k], then either \G'\ = \G"\ =0 or 

logfc-d+i|^"| <logd|a'|. 

Remark: The restriction of d is required as one can easily find an example where d = [—5—] for 
which the lemma does not hold (such examples are given in Appendix [Cj). 

Proof of Theorem^ Let Gi,G2 be the confusion graphs as defined in the theorem and assume 
the rate of the first user is alog2 d for some a G [0, 1]. The messages used can be divided into 
disjoint families J-'i,7"2, ■ ■ ■ j-^d"" according to the message for the first user. Since the first user 
can only distinguish between the letters cjj for i £ [d], we can and will assume each such family 
Ta is closed under replacing ai with aj for i > d and j G [k] . 

In order to prove this assumption is valid, it suffices to show user 1 can still distinguish between 
each of the families after these replacements. Notice that using this assumption might result in a 
different scheme, however, this shows that user 1 hasn't lost anything from this transition, while 
user 2 could possibly gain as we increased the size of each family (and he cannot lose since one 
could still use only the original families). Let Ga denote the family J^a after replacing CTj with aj 
for i > d and j G [k]. Assume by contradiction that there exist two families J-q, J-'h and two vectors 
Va ^ Ga,Vb £ Gb so that user 1 can distinguish between Fa and Fi, but cannot distinguish between 
Va and Vb- By the definition of Gi, for every coordinate z G [n], either Va[i] ^d or Vb[i\ S^^ 
or Va[i] = Vb[i] G S^^ as otherwise user 1 would be able to distinguish between them (here Vx\i] 
denotes the i'th letter in the vector v^)- Since Va G Ga and vi, £ Gb-, we know there exists Ua £ Fa 
and Ub £ Fb from which Va and Vb can be derived by the replacements above. Therefore, for 
every coordinate i G [n], either Ua[i\ S^^ or Ub[i\ S^^ or Ua[i\ = Ub[i\ £ S^^. However, this is 



in contradiction to the fact that user 1 was able to distinguish between Ta and J-f, as he cannot 
distinguish between Ua and u\,. 

Define J-"^ = J-"a n EJ^ for every Ta- Since these famiUes are pairwise disjoint, by an averaging 
argument there exists some message a for which \T'^ < d~"" • d" = S^~°'''^. By Lemma II H 
for this specific message a, \J-a\ < ^(i-")" which imphes the rate of the second user is at most 
(1 - a) log2 k. D 

Corollary 13. For the confusion graph Gi as above and any confusion graphs G2, ■ ■ ■ ,Gr, a 
feasible rate vector (a log2 d, i?2, • • • , Rr) for a £ [0, 1] must satisfy ^[=2 -Ri < (1 ~ ck) log2 k. 

Proof of TheoremlM Let Gi, G2 be the confusion graphs as defined in the theorem. Note that the 
Shannon capacity 0(^2) is precisely k — d+1, hence these rate vectors are feasible by Corollary 
m (as is also easy to see directly). Assume the rate of the first user is alog2 d for some a G [0, 1]. 
The messages used can be divided into disjoint families ^1,^2, ■ ■ ■ , Td^" according to the message 
for the first user. Since the first user can only distinguish between the letters at for i G [d], we can 
and will assume, as in the proof of Theorem [TJ that each such family Ta is closed under replacing 
(Tj with aj for i > d and j G [k] . 

Define T'a = TaCi S^ for every Ta- Since these families are pairwise disjoint, by an averaging 
argument there exists some message a for which |-Fq| < d'"^ ■ d"" = d^^~"''^ - Given the first user 
should receive the message a, the second user has at most \Ta\ = \{f {di) f {92) ■ ■ ■ fiOn) \ 9 £ J^a}\ 
different messages where /(ctj) = crjjjax{i,d} (^-s the second user can only distinguish the locations 
of (Tj for i € [k] — [d] and all other letters are indistinguishable for him) . By Lemma [T2l for this 
specific message a, \T'^\ < {k — d + l)(i~")"' which implies the rate of the second user is at most 
{l-a)log2{k-d+l)- D 

Corollary 14. For the confusion graph Gi as above and any confusion graphs G2, - - - ,Gr 5 Gi, a 
feasible rate vector (a log2 d, R2, --- ,Rr) for a £ [0,1] must satisfy ^1^2 Ri ^ (1— a) log2(A;— d+l). 

4 Two users, three letters - the complete story 

Two users and three letters is the smallest possible example of non-trivial scenario. Having only 
two letters result in each user either knowing everything or knowing nothing and obviously having 
a single user coincides with the Shannon capacity question. These smallest scenarios however 
already contain some interesting cases which we analyze using the previous results. 

Let S = {ctq, o"i, (T2} be our alphabet and let Gi, G2 be the confusion graphs of the two users 
correspondingly. If one of the confusion graphs is the complete graph, again it coincides with the 
Shannon capacity of a single graph (for the non-complete confusion graph). As stated earlier, 
there is a strong connection between the feasible rate vectors and the Shannon capacity of graphs. 
In the cases we are about to analyze, we use the fact that the Shannon capacity of every graph 
on 3 vertices which is neither the empty graph nor the clique, is precisely 2 (each such graph is 

9 



perfect, hence its Shannon capacity equals its independence number). By the symmetry between 
the users and the letters in the alphabet, it suffices to discuss only subset of the possible confusion 
graphs. 

4.1 Confusion graph with two edges 

In this subsection we show that when the first confusion graph has two edges, then no scheme can 
outperform what follows from convexity. 

Proposition 15. Let Gi = (S, {o"ocri,croO"2}), meaning the first user only distinguishes between 
the letters g\ and 02- For every G2, the optimal rate vectors are given by Corollary^ i.e. 

(a • log2 c(Gi), (1 - a) • loga 0(^2)) 

for a £ [0, 1] where log2 c{Gi) = 1. 

Remark: Note that this matches the case of a clique minus a clique for the parameters A; = 3 
and d = 2 as denoted in previous sections, but here we do not limit the confusion graph of the 
second user. 

Proof. The proof is divided into two parts, according to the intersection between the edges of Gi 
and G2. In the ffist case where there is a non-empty intersection, the bound given by Proposition 
[2] yields a maximum total rate of log2 c{Gi D G2) = 1- Therefore, one could not hope for finding 
a feasible rate vector which is not of this form (assuming G2 is not the complete graph, these are 
all optimal rate vectors as they have a total rate of 1). 

Let us assume there is no intersection between the two confusion graphs, meaning either G2 
is the empty graph or G2 = Gi. These cases match Theorems [7] and [8] respectively which indeed 
yield the desired result. D 

4.2 The first confusion graph has a single edge 

Throughout this section we denote H as the binary entropy function where given some probability 
distribution Pi,P2, ■ ■ ■ ,Pk,q (where q = l- Yli=iPi)^ 

H{pi,p2,...,Pk,q) = H{pi,p2,...,Pk) 

k 
i=l 

Proposition 16. Let Gi = (S,{o"ocri}) and G2 = (S, {(Tocr2})- The following rate vectors are 
optimal: 

. {Ri,H{R^)) for R^e [1/2,2/3]. 
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• {Ru logs 3 - i?i ) /or i?i G [2/3, log2 3 - 2/3] . 
. {H{R2),R2) for R2£ [1/2,2/3]. 

Remark: By the proposition a rate vector (Ri, R2) in the above case is feasible if and only if 

1. i?i G [0,1/2] andi?2 G [0,1] or 

2. Ri G [1/2,2/3] and R2 G [0,H{Ri)] or 

3. Ri G [2/3, log2 3 - 2/3] and R2 G [0, logs 3 - ^1] or 

4. Ri G [log2 3 - 2/3, 1] andi?2 G [0,H-^{Ri)]. 

Proof. The scenario described above is a special case of Theorem [5j Given a probability distri- 
bution p= {po,Pi,P2), Hi is distributed (po +Pi;P2), ^2 is distributed {po + P2,Pi) and {2/1,^2} is 
distributed according to p. 

The uniform distribution p = (i,i,i) yields that the rate vectors (i?i,i?2) are feasible if 
Ri + R2 < log2 3 and each Ri < H{2/3) = log2 3 — 2/3. This matches the second case described 
in the theorem, which is obviously optimal as one cannot hope to exceed a total rate of log2 3. 

By symmetry, it suffices to analyze the first case of the theorem in order to complete the 
proof. Setting pi to be some probability smaller than half bounds the rate of the second user by 
R2 < H{pi) = H{1 -pi). Moreover, the total rate Ri + R2 is bounded by H{^^, ^^,pi) = 
H{pi)+{l—pi). This shows that the rate vectors {Ri, H{Ri)) are feasible as -Ri = 1—pi < H{-^p-) 
for pi G [0, 1/2] (indeed equality holds for pi = and since H'{x) = log2(l — x) — log2 x < 2 for 
X G [1/4, 1/2], or equivalently H'{^^^) > -1 for pi G [0, 1/2], this holds for every pi G [0, 1/2] as 
well). Moreover, these rate vectors are also optimal as the bound for the total rate H(pi) + (1 —pi) 
decreases while pi increases in the section [1/3, 1/2] (using H'{x) < 1 for x G [1/3, 1/2]). D 

Remark: Although the rate vector (1, 1/2) is feasible, the vector (2", 2) is not feasible. If user 1 
needs to be able to receive 2" distinct messages, one of them has to be encoded by (o"2, o"2, • • • , (T2). 
But this has to be transmitted independently of the message of the second user, showing there is 
no (2", 2)-scheme. 

Proposition 17. Let Gi = {T,,{aoai}) and G2 be the empty graph. The following rate vectors 
are optimal: 

• (i?i, log2 3 -i?i) /or i^i G [0,/7(2/3)]. 
. {H{R2),R2) for R2 ^[1/2,2/3]. 

Remark: By the proposition a rate vector (i?i, R2) in the above case is feasible if and only if 
1. i?i G [0, log2 3 - 2/3] and R2 G [0, log2 3 - -Ri] or 
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2. Ri G [log2 3 - 2/3, 1] andfi2 G [0,H-\Ri)]. 

Proof. Our problem is monotone in the following sense. Removing an edge from one of the 
confusion graphs can only improve the feasible rate vectors. Therefore in our case, the lower 
bound of {H{R2),R2) for i?2 G [1/2,2/3] we achieved when both confusion graphs had a single 
edge can be applied here. Showing these rate vectors are also optimal will complete the proof 
as we have the trivial upper bound of log2 3 on the total rate, and by combining convexity with 
the fact that the rate vectors (i/(2/3), 2/3) and (0, log2 3) are feasible, we conclude that all other 
required rate vectors are achieved. 

Our problem is a special case of Theorem[5j Note that in the proof of Proposition [16] we showed 
an upper bound for the rates {Ri,H(Ri)) which only depended on the second user. Assuming 
the second user has rate of H{Ri) already bounds the total rate by H{Ri) + Ri. Similarly in our 
case, assuming the first user has rate of H{R2) for i?2 G [1/2,2/3] bounds the total rate of the 
two users together by H{R2) + i?2- D 

5 Conclusions and open problems 

In this work we have studied the notion of simultaneous communication in a noisy channel where 
the channel's noise may differ for each of the users. The goal is to find, for a given set of confusion 
graphs which represent the noise for each of the users, which rate vectors (or alternatively vectors) 
are feasible. As in the Shannon capacity of a channel, we care about the average rate per letter 
when the length of the messages tends to infinity. 

Our work demonstrates basic lower and upper bounds for the general case. A simple yet 
useful tool in understanding the feasible rate vectors is the convexity property which is described 
in Proposition [SJ We saw several examples where convexity and basic encoding schemes (derived 
from the Shannon capacity of the confusion graphs) are optimal. On the other hand, there are 
examples where much more can be gained by mixing the encoding for several users. The case in 
which every graph is a disjoint union of cliques is fully understood, and so is the case of 2 users 
and alphabet of size 3. Many other cases remain open. 

We conclude with several open problems it would be interesting to solve. 

• The lower and upper bounds for the maximum total rate given in this paper apply combi- 
natorial and probabilistic techniques. It would be interesting to find stronger bounds which 
possibly extend the algebraic and geometric bounds known for the Shannon capacity, such 
as the bounds given by Lovasz in [7|, Hamers [6] or Alon [1]. 

• In the non-symmetric case where we have a user whose confusion graph is a clique over k 
letters minus a clique over d letters and the other user's confusion graph is its complement, 
it would be interesting to know if Theorem [5] still holds for d > ^^- Since Lemma \TI\ does 
not hold for such d, a different approach must be used. 

12 



• It seems interesting to study graphs G for which the maximum total rate is as small as 
possible using G and G as the two confusion graphs for two users. In such a case, the upper 
bound of the Shannon capacity of the intersection (Proposition [2|) does not help as the two 
graphs are disjoint. However, by Theorem 1.1 of [1], we know there exist graphs G on k 
vertices for which both c{G) and c{G) are at most e '^^°^ '^ ^°^ '°^ *^^ . For such a graph, the 
upper bound in Proposition [1] of the maximum total rate yields 0{y/log k log log k) which is 
far less than the trivial upper bound of log2 k. 

• Most of the encoding schemes considered in this paper use randomness and therefore are not 
given explicitly. As a result, the encoding and decoding schemes are not efficient. Finding 
explicit and efficient encoding and decoding schemes for the scenarios described in the paper 
remains open. 
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A The extended Janson inequality and its application in Theo- 
rem [5] 

Below is the fuh citation of the extended Janson inequahty, fohowed by its apphcation in the lower 
bound proof of Theorem [5j 

Theorem 18 (Janson). Let S be a set, for each s £ S let pg he a real < Ps < 1- Let R 
he a random subset of S obtained by selecting each s £ S, randomly and independently, to lie 
in R with probability Ps- Let Ai,i G / be a family of subsets of S. For each i £ I, let Bi be 
the event that A-i C R. Let fi = Y2iei Pfob[Bi] be the expected number of events Bi that occur. 
Define A = ^ Prob[Bi A Bj], where the sum ranges over all ordered pairs i,j £ L,i ^ j such that 
Ai n Aj 7^ 0. Then the probability that none of the events Bi occurs is at most e~^~^ ''^. If the 
further assumption that A > // holds, this probability can also be bounded by e'^^ '^ . 

Given the process defined in the lower bound proof of Theorem [5l we fix messages ji,J2,---,jr 
for the r users. Consider the r-uniform r-bipartite hypergraph whose classes of vertices are J^ij^, 
1 < i < r. Each edge represents a consistent message, i.e. for every message x G S^ there exists 
an edge {yj G J^i,jt}ie[r] if indeed for every i £ [r], y^ = fi{x) G J^iji- Existence of some edge in 
the hypergraph indicates that this set of messages can be transmitted as required. 

Using the notations of Theorem [TSl our set S is the union of all possible messages fi{x) for 
each user i and for x G S^ which is distributed according to p. The probability of each element is 
l/rui for the relevant user i. The sets Ai represent all the consistent messages {/j(a^)}jg[rl where 
again, ?e G S^ and is distributed according to p. 

By Theorem [18] the probability that there exists no valid message as required is at most 
g-/i /2A^ Qj^g ^g^^ j^Q^ verify that indeed, as defined here, /i and A satisfy e~'^ ' < e~" '^ < 
j_ 



B Proofs of Lemmas IIOL [TTI and [12 



Proof of Lemma 03 When xi = X2 = • • • = Xf, equality holds as 

(a - 6 + l)a^°st^b + J2ie[b-i] a'"*^'""' = « • a^"^'""' 

In order to complete the proof, it suffices to show that the partial derivatives j^ are smaller on 
the left hand side than those on the right hand side for any i G [5 — 1], regardless of the values 
{xi}. Given a fixed i G [6 — 1], the derivative of the left hand side is ■^o}°^b^i = ^x^^''"' = 
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logf^a-x°^''"' . On the other hand, the derivative of the right hand side is log^ a- (^^gr^l Xi)^°S''" ^ 
which is at least as big. D 

Proof of Lemma[Tl[ We apply induction on n. For n = 1, ii Q' = Q the inequality holds as k > d 
(or both sets are empty). Otherwise there exists ai G G ioi i > d, hence |^| = |S| = A; and \G'\ = d 
for which equality holds. 

Assuming the lemma holds for any n' < n we prove it for n. Define Qi = {gig2 ■ ■ ■ dn-i \ 9 £ 
Q ^dn = o'i} and Q'j^ = t/jplS^" for every i G [k]. Note that Q'^ Q Qj and hence \Ql\ < \Q'A for every 
i > d and j G [d] (since G is closed under replacing ai with <tj for i > d and j S [A;]). In particular, 
this is true for m £ [d] so that \G'm\ = niinjg[^] \Q'A. Therefore, by the induction hypothesis and 
Lemma [10] with a = k and b = d, 

\Q\ = Ei^^i 

ie[k] 
i6[fe] 

< (A: - d)k^°^d l^™l + Y^ A:'°S'* l^^'l 

ieldi 

< {k-d + l)A:'°Sd \G'n.\ + ^ A:'°Sd ie,'l 

iG[d]-{m} 

completing the proof. D 

Proof of Lemma[TB. Again we apply induction on n. For n = 1, if ^' = ^ we have no ai G G for 
i > d and the inequality holds as \G"\ = 1 or both sets are empty. Otherwise there exists ai £ G 
for i > d, hence \G"\ = k — d + 1 and \G'\ = d for which equality holds. 

Assuming the lemma holds for any n' < n we prove it for n. Extending the previous definitions 
of Gi and G'i, let G'/ = {/(<7i)/(92) • • • /(^n-i) \ 9 e Gi} for every i e [k]. Note that G'/ C G'J and 
hence | G'/ \ < | G'j \ for every i > d and j S [A;] (since G is closed under replacing ai with ctj for i > d 
and j £ [k]). Similarly, {G'/l < | <^je[d] G'j\ for all i > d. Therefore, 

< > \G'i\ — (d — I) ■ max |^,"| 
je[d] ie[fe]-[d] 
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As before, \Ql\ < \Q'm\ for every i > d and m G [d] so that |^^| = min^gj^] |t/'. |. Therefore, by the 
induction hypothesis 

je[fe]-[(i] iG[fc]-[d] 

< (fc-d)(A;-d+iy°g''l^-l. 
By Lemma [TOl with a = k — d + 1 and b = d (which indeed satisfies 2<b<aasd< {k + l)/2), 

\g \ = 2^ \yi\ + \'^jm^i\ 

i&[k]-[d\ 

(k-d)- jd-l) ^-^ „ ^-^ „ 

^ ]—d — 2^ \g^\ + 2^\gj\ 

ie[k]-[d] je[d] 

K — a 

ie[d]-{m} 

completing the proof. D 

C An example in which Lemma [12] does not hold when d > ^y" 

Let d = 3, k = 4 and n = 2 where indeed d > ^yi = 2.5. Define 

^ = {(o-l,0-l),(o-l,Cr2),(o-l,cr3),(cri,cr4), 
((T2,cri),(cr3,CTi),(o-4,0-i)} 

which can also be viewed as {((71,(74), ((74,(71)} after replacing (74 with every (7 E S4 (as ^ has to 
be closed under these replacements). By the definitions of the lemma, 

G' = {(o-l,0-l),(o-i,0-2),(cri,CT3),(o-2,cri),(a3,cri)}, 
G" = {((73, 0-3), ((73, 0-4), ((74, 0-3)} 

and therefore the lemma does not hold as 

logfc_d+l \G"\= log2 3 > log3 5 = log^ \G'\. 



The same example can be used with larger parameters, for instance with k = 100 and d = 51 
'which is the minimal d for which d > -^y^). With these parameters, \G'\ = 2d — 1 = 101 and 
G"\ =2{k-d+l)-l = 99 and indeed logfc_rf+i \G"\ = loggo 99 > loggi 101 = log^ \G'\. 
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